TIME: Text and Image Mutual-Translation Adversarial Networks

Focusing on text-to-image (T2I) generation, we propose Text and Image Mutual-Translation Adversarial Networks (TIME), a lightweight but effective model that jointly learns T2I generator G an image captioning discriminator D under the Generative Network framework. While previous methods tackle problem as uni-directional task use pre-trained language models to enforce image--text consistency, TIME requires neither extra modules nor pre-training. We show performance of can be boosted substantially by training it with model. Specifically, adopt Transformers cross-modal connections between features word embeddings, design annealing conditional hinge loss dynamically balances adversarial learning. In our experiments, achieves state-of-the-art (SOTA) CUB dataset (Inception Score 4.91 Fréchet Inception Distance 14.3 CUB), shows promising MS-COCO downstream vision-language tasks.